Introductory Papers and Guides

Original Paper on DCA

Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006 Nov-Dec;26(6):565-74. PMID: 17099194.

General Introduction to DCA

Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ. 2016 Jan 25;352:i6. PMID: 26810254.

How to Understand a Decision Curve

Van Calster B, Wynants L, Riley RD, et al. Methodology over metrics: current scientific standards are a disservice to patients and society. Diagn Progn Res. 2019 Dec 30;3:23. PMID: 31890176.

Guide for Investigators on Reporting Decision Curves

Vickers AJ, Cronin AM, Gönen M, et al. A simple decision analytic solution to the comparison of two binary diagnostic tests. J Clin Epidemiol. 2018 Jul;99:4-10. PMID: 29596968.

Extensions to DCA

Applications to Survival Time Data among others

Vickers AJ, Elkin EB. Decision curve analysis: extensions to aid interpretation, including the clinical impact curve, time dependence, and application to individual patients. Med Decis Making. 2007 Sep-Oct;27(5):696-702. PMID: 19036144.

DCA & Treatment Response: Model Evaluation

Vickers AJ, Elkin EB. Decision curve analysis for the evaluation of treatment response prediction models. J Clin Oncol. 2007 Jun 1;25(16):2272-3. PMID: 17550609.

DCA & Treatment Response: Trial Design

Vickers AJ, Kramer BS, Baker SG. Selecting patients for randomized trials: a systematic approach based on risk group. Trials. 2006 Oct 5;7:30. PMID: 17022818; PMCID: PMC1609186.

Discussion Papers and Theoretical Background

General Theoretical Background to DCA (1)

Vickers AJ, Van Calster B. An introduction to decision curve analysis. J Clin Epidemiol. 2010 May;63(5):513-6. PMID: 19132141.

General Theoretical Background to DCA (2)

Vickers AJ, Van Calster B, Steyerberg E. Net benefit: a new approach to assessing prediction model performance. Epidemiology. 2010 Mar;21(2):260-3. PMID: 20172362.

Mathematical Underpinnings of DCA through Relationship to Relative Utility (Similar Method)

Baker SG, Cook NR, Vickers A, Kramer BS. Using relative utility curves to evaluate risk prediction. J R Stat Soc Ser A Stat Soc. 2009 Oct 1;172(4):729-748. PMID: 20069131; PMCID: PMC2804257.

Widely Cited First Prediction Model Evaluation Paper to Introduce 3-Step Approach of Discrimination, Calibration, Clinical Utility

Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010 Jan;21(1):128-38. PMID: 20010215; PMCID: PMC3575184.

Editorials and Commentaries Recommending DCA in Major Journals

Assessing the Clinical Impact of Risk Prediction Models With Decision Curves: Guidance for Correct Interpretation and Appropriate Use

Kerr KF, Brown MD, Zhu K, Janes H. Assessing the Clinical Impact of Risk Prediction Models With Decision Curves: Guidance for Correct Interpretation and Appropriate Use. J Clin Oncol. 2016 Jul 20;34(21):2534-40. PMID: 27247223; PMCID: PMC4962736.

Net Benefit Approaches to the Evaluation of Prediction Models, Molecular Markers, and Diagnostic Tests

Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ. 2016 Jan 25;352:i6. PMID: 26810254; PMCID: PMC4724785.

Decision Curve Analysis

Fitzgerald M, Saville BR, Lewis RJ. Decision curve analysis. JAMA. 2015 Jan 27;313(4):409-10. PMID: 25626037.

Beyond the Usual Prediction Accuracy Metrics: Reporting Results for Clinical Decision Making

Localio AR, Goodman S. Beyond the usual prediction accuracy metrics: reporting results for clinical decision making. Ann Intern Med. 2012 Aug 21;157(4):294-5. PMID: 22910942.

Assessing Performance and Clinical Usefulness in Prediction Models With Survival Outcomes: Practical Guidance for Cox Proportional Hazards Models

David J. McLernon, Daniele Giardiello, Ben Van Calster, et al; topic groups 6 and 8 of the STRATOS Initiative. Assessing Performance and Clinical Usefulness in Prediction Models With Survival Outcomes: Practical Guidance for Cox Proportional Hazards Models. Ann Intern Med.2023;176:105-114. [Epub 27 December 2022].

Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review

de Hond AAH, Leeuwenberg AM, Hooft L, et al. Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review. npj Digit Med. 2022 Jan 20;5(1):2. doi: 10.1038/s41746-021-00549-7. PMID: 35348862; PMCID: PMC8393232.

Other Editorials

Decision curve analysis in the evaluation of radiology research

Vickers AJ, Woo S. Decision curve analysis in the evaluation of radiology research. Eur Radiol. 2022 Sep;32(9):5787-5789. doi: 10.1007/s00330-022-08685-8. Epub 2022 Mar 29. PMID: 35348862.

Decision curve analysis to evaluate the clinical benefit of prediction models

Vickers AJ, Holland F. Decision curve analysis to evaluate the clinical benefit of prediction models. Spine J. 2021 Oct;21(10):1643-1648. doi: 10.1016/j.spinee.2021.02.024. Epub 2021 Mar 3. PMID: 33676020; PMCID: PMC8413398.

Is My Clinical Prediction Model Clinically Useful? A Primer on Decision Curve Analysis

Mijderwijk HJ, Nieboer D. Is My Clinical Prediction Model Clinically Useful? A Primer on Decision Curve Analysis. In: Staartjes VE, Regli L, Serra C, editors. Machine Learning in Clinical Neuroscience. Acta Neurochirurgica Supplement, vol 134. Springer, Cham; 2022. p. 115-118. doi: 10.1007/978-3-030-85292-4_15.

Is My Clinical Prediction Model Clinically Useful? A Primer on Decision Curve Analysis

Mijderwijk HJ, Nieboer D. Is My Clinical Prediction Model Clinically Useful? A Primer on Decision Curve Analysis. Acta Neurochir Suppl. 2022;134:115-118. doi: 10.1007/978-3-030-85292-4_15. PMID: 34862535.

Moving beyond AUC: decision curve analysis for quantifying net benefit of risk prediction models

Sadatsafavi M, Adibi A, Puhan M, Gershon A, Aaron SD, Sin DD. Moving beyond AUC: decision curve analysis for quantifying net benefit of risk prediction models. Eur Respir J. 2021 Nov 4;58(5):2101186. doi: 10.1183/13993003.01186-2021. PMID: 34735641.

Newly Added

A Systematic Review of the Literature Demonstrates Some Errors in the Use of Decision Curve Analysis but Generally Correct Interpretation of Findings

Capogrosso P, Vickers AJ. Med Decis Making. 2019 Jul;39(5):493-498. PMID: 30819037.

A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models

Christodoulou E, et al. J Clin Epidemiol 2019;110:12–22.

A framework for making predictive models useful in practice

Jung K, et al. J Am Med Inform Assoc 2021; 28 (6): 1149–58.

Calculating net benefit under resource constraints

Karandeep Singh, Nigam H Shah, Andrew J Vickers, Assessing the net benefit of machine learning models in the presence of resource constraints, Journal of the American Medical Informatics Association, Volume 30, Issue 4, April 2023, Pages 668–673, https://doi.org/10.1093/jamia/ocad006

R code for calculating net benefit under resource constraints

Assessing Performance and Clinical Usefulness in Prediction Models With Survival Outcomes: Practical Guidance for Cox Proportional Hazards Models

McLernon DJ, et al. Ann Intern Med. 2023 Jan;176(1):105-114. PMID: 36571841.

Seven Common Errors in Decision Curve Analysis

Overinterpretation of findings in machine learning prediction model studies in oncology: a systematic review

Dhiman P, Ma J, Andaur Navarro CL, Speich B, Bullock G, Damen JAA, Hooft L, Kirtley S, Riley RD, Van Calster B, Moons KGM, Collins GS. Journal of Clinical Epidemiology. 2023; ISSN 0895-4356. https://doi.org/10.1016/j.jclinepi.2023.03.012.